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Abstract: Let M be a compact Riemannian submanifold of R m of dimension d 
and let X\, . . . , X n be a sample of i.i.d. points in M with uniform distribution. 
We study the random operators 

n 

Ajh.,„/(p) := —^+2 V K(^=^i)(f(Xi) - f(p)), p G M 

i — l 

where K(u) : = hnw£~^ u ^ ^ 4 is the Gaussian kernel and h n — * as n — > oo. 

Such operators can be viewed as graph laplacians (for a weighted graph with 
vertices at data points) and they have been used in the machine learning 
literature to approximate the Laplace-Beltrami operator of M, Am/ (divided 
by the Riemannian volume of the manifold). We prove several results on a.s. 
and distributional convergence of the deviations A h n f(p) — f^Ajvf/(p) for 

n ' I M 

smooth functions / both pointwisc and uniformly in / and p (here \fi\ = /i(M) 
and fi is the Riemannian volume measure). In particular, we show that for any 
class T of three times diffcrcntiable functions on M with uniformly bounded 
derivatives 



sup sup 

peM fer 



&h n ,pf(p)~ T^^Mftp) 



()l /io g (i/M 



nil 



d+2 



as soon as 

n/ i f+ 2 /log/i,7 1 -> oo and nh^+ 4 / log h' 1 -» 0, 

and also prove asymptotic normality of A; lji P /(p) — j^-Am/(p) (functional 
CLT) for a fixed p £ M and uniformly in /. 



1. Introduction 

Recently, there have been several developments in statistical analysis of data sup- 
ported on a submanifold in a high dimensional space based on the idea of approx- 
imation of the Laplace-Beltrami operator of the manifold (and some more general 
operators that contain information not only about the geometry of the manifold, 
but also about the unknown density of data points) by empirical graph Laplacian 
operators. If V is a finite set of vertices and W := {wij)i^j^v is a symmetric non- 
negative definite matrix of weights with Wij > ("adjacency matrix"), then the 
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graph Laplacian of the weighted graph (V, W) is defined as the matrix (operator) 
L = D — W, where D is the diagonal matrix with the degrees of vertices 

deg(i) := ^Wy, ieV 

on the diagonal. Such (unnormalized) graph Laplacians along with their normalized 
counterparts L := I — D~ 1 / 2 LD~ 1 / 2 have been studied extensively in spectral 
graph theory. If now X\, ■ ■ ■ , X n are i.i.d. points uniformly distributed in a compact 
Riemannian submanifod M of R m of dimension d < m, it has been suggested in the 
literature to view {X\, . . . , X n } as the set V of vertices of the graph and to define 
the weights as Wij x e~" Xi " Xj " ^ 4h with a small parameter h > 0, to approximate 
the Laplace-Beltrami operator Am of M, Am(/) = div(grad(/)). More precisely, 
the estimate is defined as 

i n Y 

where (m) := ^ 4 ^ d/2 e~N"N / 4 is the Gaussian kernel and /i n — * as n — » oo (if the 
functions / are restricted to V, this can be viewed, up to a sign, as a graph Laplacian 
operator). We will call such operators empirical graph Laplacians and their limit 
as n — > oo on smooth functions / is j^Ajv//(p), where \fi\ is the Riemannian 
volume of M. There are numerous statistical applications of such an approximation 
of the manifold Laplacian by its empirical version. In particular, one can look 
at projections of the data on eigenspaces of the empirical Laplacian Ah n , n (the 
technique sometimes called diffusion maps) in order to try to recover geometrically 
relevant features of the data (as in the method of spectral clustering) or use the 
kernels associated with this operator to approximate the heat kernel of the manifold 
and to use it to design kernel machines suitable, for instance, for classification of 
the manifold data. 

Convergence properties of empirical graph Laplacians have been first studied 
by Belkin and Niyogi [H and Hein, Audibert and von Luxburg Q. Our goal in 
this paper is to provide a more subtle probabilistic analysis of such operators. In 
particular, for proper classes of smooth functions T and for a fixed p € M, we 
establish a functional CLT for \J nhn +2 (A hntP f(p) - AA M /(p)), / e T, and also 
show that 



sup sup 



&h n ,pf(p) - ^-A M f{p) 




iog(i/M 



i h 



d+2 



a.s. 



(under suitable assumptions on h n ). The asymptotic properties of empirical lapla- 
cians are closely related to the well developed theory of kernel density and kernel 
regression estimators, which can be viewed as examples of so called local empirical 
processes, as in Q. Our proofs are essentially based on an extension of this type of 
results to the case of data on the manifolds (for kernel density estimation on man- 
ifolds, see, e.g., and references therein). For simplicity, we are considering in 
the current paper only uniform distributions on manifolds and Gaussian kernels K, 
but more general types of operators that occur in the case when the distribution of 
the data is not uniform and more general kernels (as in the paper of Hein, Audibert 
and von Luxburg can be dealt with quite similarly using the methods of the 
paper. 
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2. Some geometric background 

We refer to [3| for the basic definitions and notations from Riemannian geometry. 
Given a manifold M and p £ M, T P (M) will denote the tangent space to M at p, and 
TM the tangent bundle. Let M be a complete connected (embedded) Riemannian 
submanifold of R m , of dimension d < m, meaning that M is a complete connected 
Riemannian manifold and that the inclusion map </> : M <— > R m is an isometric em- 
bedding, that is, (i) <p is differentiable and injective, (ii) d<fi p : T p (M) i— > T0( p )(R m ) 
is an isometry onto its image, Tm p \((()(M)), and (hi) <p is a homeomorphism onto 
4>{M) with the topology inherited from R m . When no confusion may arise, we iden- 
tify M with 4>{M). M being complete, by the Hopf and Rinow theorem (e.g., [3], 
p. 146) the closed bounded sets of M are compact. 

Given p £ M and v £ T p (M), let j(t,p 7 v), t > 0, be the geodesic starting 
at p with velocity v, 7(0, p,v) = p and j'(0,p,v) = v. The exponential map E p : 
T p (M) f — * M (the usual notation is exp p ) is defined by £ p {v) — j(l,p,v). This map 
is defined on all of T p {M) by the Hopf and Rinow theorem. 

A normal neighborhood V of p £ M is one for which a) every point q in V can be 
joined to p by a unique geodesic 7(i,_p, v), < t < 1, with 7(0) = p, 7(1) = (7 and b) 
the exponential map centered at p, £ p , is a diffeomorphism between a neighborhood 
of £ T p (M) and 1/. If B C V is a normal ball of center p, that is, the image by the 
exponential map of a ball around zero in T p (M), then the unique geodesic joining 
p to q £ B is a minimizing geodesic, which means that if c?m denotes the distance 
in M and | • | denotes the norm of T p {M) defined by the Riemannian structure of 
M, then dM(p,£ P (v)) — \v\. Given an orthonormal basis ei, . . . ,e<j of T p (M), the 
normal coordinates centered at p (or the p-normal coordinates) of q £ V are the 
components qf = (£~ 1 (q),ei) of E p 1 (q) in this basis. (The super-index p will be 
omitted when no confusion may arise, but we will need it when considering normal 
coordinates based at different points.) Every point in M has a normal neighborhood. 
See 0], Propositions 2.7, and 3.6, pp. 64 and 70 for these facts. Actually, more is 
true (e.g., [4J], Theorem 3.7 and Remark 3.8): 

Proposition 2.1. For every p £ M there exist a neighborhood W ofp and a number 
5 > such that: (a) for every q in W , £ q is defined on the 8 ball around £ T q (M), 
B s (0) C T q {M), and £ q (B s (0)) is a normal ball for q, (b) W C £ q (B 5 (0)) (W is a 
normal neighborhood of all of its points ), and (c) the function 

F{q,v) := (q,£ q (v)) 

is a diffeomorphism from Wg :— W x B$(0) = {(q, v) £ TM : q £W,\v\ < 6} onto 
its image in M x M and \dF\ is bounded away from zero on W$. 

Such a neighborhood W of p is called totally or uniformly normal. In partic- 
ular, £ q (v) is jointly differentiable in (q,v) £ Ws if W is a uniformly normal 
neighborhood. Moreover, for every q £ W and v £ T q {M) such that \v\ < 6, 

dM(q,£ q (v)) = \v\. 

Remark 2.1. By shrinking W and taking 8/2 instead of 8 if necessary, we can 
assume in Proposition 2.1 that the closure of W and the closure of Ws (which 
are compact because M is complete) are contained in W and W' s , satisfying the 
properties described in the previous proposition. Moreover, we can also assume that 
for all q in W, £ q (Bs(0)) is contained in a strongly convex normal ball around p 
(these points are at distances less than 28 from p, so this assumption can be met by 
further shrinking W and taking a smaller 8 if necessary, since every point in M has 
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a strongly convex geodesic ball, e.g. Proposition 4.2 in do Carmo, loc. cit.; strongly 
convex set: for any two points in the set, the minimizing geodesic joining them lies 
in the set). We will assume without loss of generality and without further mention 
that our uniformly normal neighborhoods W satisfy these two conditions. 

Let W be a uniformly normal neighborhood of p as in the remark, let W be 
a uniformly normal neighborhood of p containing the closure of W, and let < 
S < 5' be as in the proposition and the remark. Let us choose an orthonormal 
basis ei, . . . , e<j of T p {M) and define an orthonormal frame e\ , . . . , e d , q € W, by 
parallel transport of ei, . . . , e<j from p to q along the unique minimizing geodesic 
joining p and q. So, e\ , . . . , e 9 d is an orthonormal basis of T q {M) for each q e W . 
This frame depends differentiably on q as parallel transport is differentiablc (and 
preserves length and angle). So, we have on W a system of normal coordinates 
centered at q for every q <E W' , namely, if x G £ q (Bs>(0)) is x = £ 9 (X^=i w » e i )' 
then the coordinates of x, x\ are x\ — Vi, the components of £~ 1 (x). Let now / be 
a differentiable function / : M i— > R, and define / : W' s , ^ R by 

f(q,v) := ffa(F(q,v)) = f(£ q (v)), 

where TT2 is the projection of M x M onto its second component. This map is 
differentiable by the previous proposition. In particular, if we take as coordinates 
of (q, v) e Wg, the normal coordinates centered at p of q, (q\, ...,%) = (gf, . . . ,q p d ) 
and for v G T q {M) the coordinates v\, . . . Vd in the basis e\ , . . . , ei, which coincide 
with the normal coordinates centered at q of £ q {v), then the real function of 2c? 
variables (which we keep calling /; the same convention applies to other similar 
cases below) 

f(q 1 ,...,q d ,v 1 ,...,v d ) = f(q,v) 

is differentiable on the preimage of W' s , by this system of coordinates. Moreover, 
by compactness, each of its partial derivatives (of any order) is uniformly bounded 
on the preimage of Wg. If we denote by x\ the normal coordinates centered at q, 
we obviously have that for each r € N and . . . ,i r ) € {1, . . . , d} r , 

d r f d r f 
dxTdxJ 2 . . . dxl {X) = dv n dv l2 . . . dv lr («!' ■ - 9* *!>- ' *S)- 

We then conclude that 

each of the partial derivatives (any order) of f with respect to the 

(2.1) 

q — normal coordinates xf is uniformly bounded in q € W and x e £ q (Bs(0)). 

In particular, the error term in any limited Taylor development of / in g-normal 
coordinates can be bounded uniformly in q for all \v\ < S, that is, if P^(x\, . . . , x q d ) 
is the Taylor polynomial of degree k in these coordinates, we have, for q e W and 
\£ q \x)\ < 5, 

(2.2) \f(x)-I»(x\,...,xl)\ <C k (d M (q,x)) k +\ 

where Ck is a constant that depends only on k. Moreover, the coefficients of the 
polynomials are differentiable functions of q, in particular bounded on W. The 
q- uniformity of these Taylor developments for \v\ < S and the g-differentiability 
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of their coefficients will be very useful below. We will apply these properties to 
the canonical (in R m ) coordinates of the embedding <j> and also to the functions 
cif - )^)' wnere x i = x \ are the p- normal coordinates. 
In what follows, we often deal with classes T of functions on M whose partial 
derivatives up to a given order k are uniformly bounded in M or in a neighborhood 
U of a point p £ M. In such cases, we say that T is unformly bounded up to the 
k-th order in M (or in U). Clearly, this property does not depend on the choice 
of normal (or even arbitrary) local coordinates. In the case when we choose an 
orthonormal frame e\ , . . . , e d and define normal coordinates and the corresponding 
partial derivatives as described above, we can also deal with continuity of partial 
derivatives. We say that T is a uniformly bounded and equicontinuous up to the k-th 
order class of functions iff there exists a finite covering of M with uniformly normal 
neighborhoods such that, in each neighborhood, the sets of partial derivatives of 
any order < k are uniformly bounded and equicontinuous. This definition does not 
depend on the choice of orthonormal frames in the neighborhoods. Such classes are 
useful because the remainders in Taylor developments are uniform both in q e M 
and in / e T . 

Consider now, for q e W and x e £ q (B$> (0)), the tangent vector fields -^(x), 
i = 1 , . . . , d, and simply write (x) for j—? (x) . Taking the previous coordinates 

qi,Vj in Wg,, denote by Xi(£q( v )) = Xi(<7i, ■ ■ ■ ,Qd,Vi, . . - ,v d ) the p-normal coordi- 
nates of £ q {v), which are differentiable. By the chain rule, 




Hence, if g1j(x) are the components of the metric tensor at x in g-normal coordi- 
nates, we have 




By (2.2), we conclude that if P^{x\, . . . , x q d ) is the Taylor polynomial of degree k in 
the expansion of gfj (x) in g-normal coordinates, then there are constants C\ that 
depend only on k such that, for all q e W and x € £ q (Bs(0)), 

(2-3) \g^x) - P q k {x\, . . . , x\)\ < C k (d M (q, x)) k+1 . 

This will also be useful below. These remarks allow us to strengthen several results 
based on Taylor expansions by making them uniform in q, as follows. 

Proposition 2.2. Given p e M , let W and Ws be as in Remark 2.1, and consider 
for each q G W , the q-normal system of coordinates defined above. Then, 
(a) for every q <G W the components g q ^(x\, . . . , x q d ) of the metric tensor in q-normal 
coordinates admit the following expansion, uniform in q and in x G £ q (Bs(0)) 
(B 5 (0) G T q {M)): 

(2.4) g*.{x\, ...,x q d ) = 8 l3 - \R q rso {0)x q x q + 0{d? M {q,x)), 

(Einstein notation) where R q rs -{Qi) are the components of the curvature tensor at 
q in q-normal coordinates, and, as a consequence, the following expansion of the 
volume element is also uniform in q and x: 

(2.5) ^det(g q 3 )(xl ...,x q ) = l- ^Ric^O)^ + 0(d 3 M (q, x)), 
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where Ric^ s (0) are the components of the Ricci tensor at q in q-normal coordinates. 

(b) There exists C < oo such that for all q £ W and x £ £ q (Bg(0)), 

(2.6) < d 2 M (q,x) - \\</>(q) - 4>(x)f < Cd 4 M (q,x). 

(c) For each 1 < a < m, the a-th component in canonical coordinates of R m of 
4>(£ q (v)), <fi a (£ q (v)), admits the following expansion in q-normal coordinates Vi of 
£ q (v), uniform in q £ W and \v\ < 8, 

(2.7) <t> a (£ q (v)) - c/> a (q) = ^fMK + l-^rM,0)viVj + 0(\v\ 3 ), 

a = 1, . . . ,m, where <p(q, v) — tf>(£ q (v)). 

Note that Y^t=i ^r(q^) v i are the R m -canonical coordinates centered at 4>(q) 
of the vector d(f> q (v) £ T^ q ^{(j){M)) C R m since g?j(q) = Sij. Hence, if we identify 
the tangent space to <j>{M) at 4>(q) with an affine subspace of R m , part c) says 
that the difference between cf>(£ q (v)) £ R m and the tangent vector to the geodesic 
<fr(l(t, q, v)) at 4>(q) (t = 0), (f)(q) + d<j> q {v), is a vector of the form 



d 2 K 



iq,0)viv 3 :a = l,...,m\ + 0(\v\ 3 ) 



where 0(|w| 3 ) is uniform in |t>| < 6 and q. Ignoring the embedding, this gives an 
expansion of the exponential map as 

(2.7') £ q {v) =q + v + Q q (v,v) + 0(\v\ 3 ) 

uniform in q £ W and \ v\ < S, where Q q is a R m -valued bilinear map on T q (M) (ac- 
tually, on 70(g) (0(M))) that depends differentiably on gr, hence uniformly bounded 
in qeW. 



Proof of Proposition 2.2. (a) follows from the expansions of and ydet(g^) in 

q-normal coordinates (e.g. in [l2j], p. 41), the expansion of its determinant (e.g., 
[12j |. p. 45), and the uniformity provided by (2.3). 

(c) follows by direct application of (2.2) to / = 4> a , a = 1, . . . ,m. 

(b) Following Smolyanov, Weizsacker and Wittich [l3j, for q £ W and x = £ q (v), 
\v\ < 6, and applying (2.2) for / = <j> a , a = 1, . . . , m, we have 

Q< <P M (g,x)- ||0( g ) -0(x)|| 2 _ \v\ 2 -E«=i (M£ q (y)) - Mq)) 2 

di[(q,x) |u| 4 

M 4 

+ 0(\v\), 

where the term 0(|w|) is dominated by C±\v\ for a constant C4 that does not de- 
pend on q or v. But now, continuing the proof in this reference, which consists in 
developing and simplifying the ratio above, we obtain that, uniformly in q £ W , 
x = £ q (v), \v\ < 5, 

dl I (q,x)~U(g)~^x)r _ 1 ^ (^fefaO)^) 2 
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and note also that, by compactness, the main term is bounded by a fixed finite 
constant in this domain. □ 

Although we have been using [4| as our main reference on Riemannian geome- 
try, another nice user-friendly reference for the exponential map in particular and 
Riemannian manifolds in general is 0]. We thank Jesse Ratzkin for reading this 
section and making comments (of course, any mistakes are ours). 



3. Approximation of the Laplacian by averaging kernel operators 

Let M be a compact connected Riemannian submanifold of R m , m > d (if M 
is compact, it is automatically embedded, that is, conditions (i) and (ii) on the 
immersion 4> imply that is a homeomorphism onto its image). [ See a remark at 
the end of this section for a relaxation of this condition.] Let \i be its Riemannian 
volume measure and |/i| = fi(M). Let K : R m i— > R be the Gaussian kernel of R m , 

where is the norm of x in R m . Let A be a random variable taking values 
in M with law the normalized volume element, fi/\(Jt\, and let / : M i— > R be 
a differentiable function. The object of this section is to show that the Laplace- 
Beltrami operator or Laplacian of M , 

Am/0) = div S rad (/)(p) 
(in coordinates, A M (f) = -^J= ) ^- (g*3 y'det (ff^J §f. ) , where (gV) = (.gy) _1 ) 

can be approximated, uniformly in / (with some partial derivatives bounded), and 
in p G M, by the averaging kernel operator 



(3.2) A hn /(p) := 



with rates depending on h n — > 0. Note that, by the expansion (2.4) of the metric 
tensor in normal coordinates centered at p, we have, in these coordinates, 



(3-3) M(P) = L^W' 

i= 1 1 

(where p = (0, . . . , 0) in these coordinates). 

With some abuse of notation, given p G M, we denote the derivatives with 
respect to the components of v of f(p,v) = f o £ p (i>) at (p,i>), w = Ep l {x), by 

/'(x), /"(x), etc. (so, for instance, if x = £ p (v), f'(x) = (§^(p,v), §^(p,v))) 

(in fact, f( k \x) depends on p and therefore it should have been denoted fp k \x), 
but in the context we are using this notation p is typically fixed, so, we will drop 
p, hopefully, without causing a confusion). 

Theorem 3.1. We have, for any p, any normal neighborhood U p of p and a class 
T uniformly bounded up to the third order in U p , that 

(3.4) sup A h J(p) - ^A M /0) =0(h n ). 

f£F iMl 
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as h n 0. Moreover, for any class of functions uniformly bounded up to the third 
order in M, 



(3.5) 



as h n — ► 0. 



sup sup 

f£Fp£M 



A fc „/(p) - nA M /(n) - 0(/i„). 



Proof. M being regular, the embedding <j> is a homeomorphism of M onto 4>{M), 
and M being compact, the uniformities defined respectively on M by the intrinsic 
metric <1m(PiQ) and by the metric from R m , dR.m(p, q) := ||0(p) — <p{q)\\ coincide 
(e.g., Bourbaki (1940), Theorem II. 4.1, p. 107), that is, given e > there exists 
S > such that if c/m(p, q) < 5 for p, q e M, then dn,m(p, g) < e, and conversely. 
Hence, in Proposition 2.2, we can replace B s (0) C T 9 (M) by B' s ,(0) := £ 9 -1 -{x € 
M : ||(/>(q) — 0(x)|| < <5'} for some 6' depending on S but not onp or q. From here on, 
we identify M with <j>(M) (that is, we leave <j> implicit). Let p £ M. Given h n \ 0, 
let 

(3.6) B n := {x e M : \\p - x\\ < Lh n (log /i" 1 ) 1 / 2 } 

for a constant L to be chosen later. As soon as Lh n (\ogh~ 1 ) 1,/2 < 8' , the neighbor- 
hood of e T p {M), 

E — f~ x B 

is well defined, and, by (2.6), since \v\ — dM{p,£ P (v)), we have on B n that 

H 2 >lb-^(«)|| 2 >H 2 (i-cH 2 ) 

with C independent of p e M. Hence, for all n> N 0} for some N < oo independent 
of p, we have 



(3.7) 



{v e T p M : |v| < LhnQogh- 1 ) 1 ' 2 } C B n 
C e T p M : |v| < 2Lh n (\ogh- 1 ) 1 / 2 }, 



where the coefficient 2 can be replaced by A„ — > 1. Assume n> N . 
By the definitions of X and B n , 



E 



K 



(3.8) 



< 



< 



P-X 

h n 
2II/IU 

( 47r )d/2 
2|lflloc 



(/(X)-/W)/(l€M\B„ 

e -||p-x|| 2 /4^^) 
M\B n ImI 



(47r) d / 2 " 



Taking into account that the measure /z has density y^det^j) in p-normal coordi- 
nates (hence on B n ), we have 



E 
(3.9) 



K 



P-X 



(f(X) - f(p))I(X G B n ) 



(4 



^7J^ / g e-^- £ ^ 2 /* h n(f(£ p (v)) f(£ p (0)))y/dct( gij )(v)dv. 



With the notation introduced just before the statement of the theorem, the Taylor 
expansion of / in p-normal coordinates can be written as 

/(£») - f(£ p (0)) = (f'(p),v) + \f"{p){v,v) + ±f"\Z v )(v,v,v). 
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where = £ p (9 v v) for some 8 V G [0, 1]. Next we will estimate the three terms that 
result from combining this Taylor development with equation (3.9). Recall that, by 
Proposition 2.2, there are C\ and C independent of p such that 



(3.io) ^dct( giJ )(v) < i + cm, ^ H - C H 4 < lb - £ P {vW < l< 

for v e B n , and recall also (3.7) on the size of B n . Using these facts and the 
development of the exponential about — \v\ 2 /Ah\ immediately gives 



/, 



||p-£ p („)|| 2 /4fci 



'B n 

where 



■(f'(p),v}^det( 9ij )(v)dv = J_ e-^ 2 / 4 ^(f\p),v)dv + R n 



\R n \ < [ ( e -(M 2 -OI 4 )/< _ e-^ h ')\f'(p)\\v\dv 



Ci 



L 



B„ 



< I e 

>B n 

<hl +d I e 
= D\f'(p)\hl+ d , 



*-|/'(p)|(qt;| 5 /(4^) + Ci|t,| 3 )d« 
l " |2/8 |/'(j>)l(C|^| 5 /4 + Ci^l 3 )^ 



and D only depends on C, C\ and d. Moreover, since B c n C \\v\ > Lh n (\og h n 1 ) 1 ^ 2 } 
and 

f e-W 2 ^(f(p),v}dv = Q, 

we also have 



/ 

Jb, 



<(f'(p),v)dv 



(f'(p),v)dv 



< 



\f(p)\ I 

J\v 



\v\>Lh n {\og h' 1 ) 1 / 2 



= \f'(p)\h 



l+d 



lul^LOogfc- 1 ) 1 /^ 



^'<\v\dv 
-^ 2 / 4 \u\du 



= C d \f'(p)\h 1 + d f e- r ' A / 4 r d dr 

JryLilogh.- 1 ) 1 / 2 

<c> d \f\p)\L d -ihi+ d + L2 ^iogh^r-^\ 

where Cd and C' d are constants depending only on d. Collecting terms and assuming 

L 2 /4 > 2, 

we obtain 

p -\\p-£ v (v)\\ 2 /^lirn 



(3.11) 



>{f'(p),v)^/det( gij )(v)dv\ < D 2 \f(p)\h 3 + d , 



for all n> N , and where D 2 does not depend on p. 
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The remainder term is of a similar order if |/"'| is uniformly bounded in a 
neighborhood of p: if c is such a bound, 



(3.12) 



e -\\ P -e p (v)\\ 2 /4h 



B, 



f'"(Z v )(v,v,v)yJdet( gij )(v)dv 
< c ( e-l^/^l^l^l + d^log/i- 1 !^ < D 3 ch 3 + d , 



where D 3 docs not depend on / or p (as long as n > No). 

Finally, we consider the second term, which is the one that gives the key rela- 
tionship to the Laplacian. Proceeding as we did for the first term, we see that 



(3.13) 



J e -\\p-£ P (v)\\ 2 /4hl f/{p) (U; v) ^~) {v)dv 

= h d+2 f e-M 2 ^.f(p)(v,v)dv + R n , 



where now 

(3.14) \R n \ < D 4 \f'(p)\hi+ d + D 5 \f"{p)\hl+ d+L2 l\\ogh- 1 ) d / 2 < D,\f"{p)\hi +d 
if 

L 2 /A > 2 

and n> No, where the constants D do not depend on / or p. Now, by definition 

™-l/.W>-(^(0))' =(£k^ ■ 

so that, on account of (3.3), 

(3-15) d l ~ x 

= 2(An) d ' 2 Y. ^(P) = 2(4n) d ^A M f(p). 

i=l * 

Combining the bounds (3.11), (3.12), (3.13)-(3.14) and the identity (3.15) with 
(3.9), we obtain the first part of the theorem. Note that we need to choose L such 
that L 2 /4 > 2 and then iV such that LhN a (\ogh~ N 1 a ) 1 /' 2 < 8, and that with these 
choices the bounds obtained on the terms that tend to zero in the proof depend 
only on the sup of the derivatives of / and on the sup of certain diffcrcntiablc 
functions of (q, v) on W$, where W is a uniformly normal neighborhood of p and 
S the corresponding number from Proposition 2.2 and Remark 2.1. These bounds 
are the same if we replace p by any q € W by Proposition 2.2. M being compact, 
it can be covered by a finite number of uniformly normal neighborhoods Wj, i < k, 
with numbers Si as prescribed in Proposition 2.1 and Remark 2.1. Taking S to be 
the minimum of Si, i = 1, . . . , fc, and the constants in the bounds in the first part 
of the proof as the maximum of the constants in these bounds for each of the k 
neighborhoods, the above estimates work uniformly on q e M, giving the second 
part of the theorem. □ 
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Remark 3.1. (1) Obviously, the first part of the theorem, namely the limit (3.4), 
does not require the manifold M to be compact. (2) If instead of assuming existence 
and boundedness of the third order partial derivatives in a neighborhood of p we 
assume that the second order derivatives are continuous in a neighborhood of p, 
then we can proceed as in the above proof except for the remainder term (3.12), 
that now can be replaced by 



(3.16) 



-\\v-e v m 2 /^l {f ^ v) _ f"(p))( v> v)y/det( gij )(v)dv 



o{hl +d ). 



Hence, in this case we still have 

(3.17) A h J{p)^ -^-A M f(p) as h n -» 0. 

A similar observation can be made regarding (3.5). 

Remark 3.2. Suppose N is a compact Riemannian ci-dimensional submanifold of 
R m with boundary (for the definition, see (l2| . p. 70-71). The Riemannian volume 
measure n is still finite. Then, Theorem 3.1 is still true if X is a N- valued random 
variable with law n/\n\ with = n(N), and M a compact subset of N interior to 
N. The proof is essentially the same. 

The first part of Theorem 3.1, without uniformity in /, is proved in a more 
general setting in ||. 

Theorem 3.1 provides the basis for the estimation of the Laplacian of M by 
independent sampling from the space according to the normalized volume element, 
which is what we do for the rest of this article. 



4. Pointwise approximation of the Laplacian by graph Laplacians 

Let M be a compact Riemannian submanifold of R d (or, in more generality, let M 
be as in Remark 3.2), and let X,Xi, i G N, be independent identically distributed 
random variables with law . The 'empirical counterpart' of the averaging kernel 
operator from Section 3 corresponding to such a sequence is the so called graph 
Laplacian 

(4.1) A hn , n f(p) := * £ A'fc^i) (f(X t ) f(p)), 



nh d + 2 ^ V K 

with K given by (3.1) (other kernels are possible). 

We begin with the pointwise central limit theorem for a single function /, as a 
lemma for the CLT uniform in /. 

Proposition 4.1. Assume f has partial derivatives up to the third order continuous 
in a neighborhood of p. Let h n — > be such that nh^ — > oo and nhf^^ — > 0. Then, 



i 4.2 ) \Jnht +2 \A hn . n f( P ) - iA M /( P ) 

L \n\ 



sg in distribution, 



where g is a standard normal random variable and 

" 2= UttWh] J Ri 6 H /2 (X^ ( ^) dv = 25(2^72pS| 



(4.3) s 
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Proof. Since by Theorem 3.1, 



n/4 +2 (A M /(p) - &hj(p)) =0i^/nh d n +4 
it suffices to prove that the sequence 

-ek( p - x 



(4.4) 



(/(*)-/(?)) 



is asymptotically centered normal with variance s 2 . To prove this we first observe 
that we can restrict to X t <G B n because, as in (3.8), 



(4.5) 



h d n +2 



E 



K 2 f ?-J^- ) (J( Xi ) - f(p)) 2 I(X eM\B n ) 



< mi r 

- (4ir) d h d + 2 Jm ' 



if we take L 2 /2 > d+2. Now, on the restriction to B n we replace f(Xi) — f(p) by 
its Taylor expansion up to the second order plus remainder, as done in the proof of 
Theorem 3.1. The second term and the remainder parts, namely 

^E {K(^^y ,l {p){£-\X i ),£-\X i ))I{X i e B n ) 



Z n a ~ \lnh d + 2 
(4.6) 



EK 



(^-y"{jp){£-\X),£-\X))I{X e B n ) 



and 



i=l 



(4.7) 



x {E-^Xi^E-^X^E-^Xi^IiXi € B n ) 



-EK{^^\f''{0{^\X),£-\X),£;\X))I{XeB n ) 



tend to zero in probability: the estimates (3.10) give 
1 



(4.8) 



< 



(4n) d \n\h d n +2 j,j 
2 



< 



{^Y\^\h d + 2 Jg n 

2h d + 4 



e" l|2/2ft " (/" (p) («, «)) 2 v /det( 3y )(«) ( i« 

e-l"l 2 / 3 ^|/"(p)| 2 |t;| 4 ^ 
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and, with c = sup xeU \f"'(x)\, 



< 



(4.9) 



< 



{4ir) d / 2 \ t i\Vnh d + 2 
3cn 

(4ir) d /2\v\Vnht +2 JB 
3hl+ d cn 



e -\\ P -e P (v)e/4hl\ v] 3 J det ^ g ..^ v ) dv 

e -H 2 /5hl^3 dv 



-\v\- 



(47r) rf / 2 | M |V^n +2 JR d 
0. 



/ 5 |w| 3 di; 



= 0[^nh d n +A 
Finally, we show that the linear term part 



Z n ,i : = V n h 



,d+2 



(4.10) 



EK 



p-X 



(f'( P ),£- 1 (X l ))I(X l eB n ) 
{f{ P ),£- l {X))I{XzB n ) 



is asymptotically N(0, s 2 ). Since, by (3.11), 



\fw^ EK {h^) {f,(p) '^ lix))Iix e 6n) 

= ^o(^ 3 ) = o(v / ^)-o, 



and since, by computations similar to the ones leading to (3.11), 

-1 2 



1 



,d+2 



E 



K 



p-X 



1 



{f(jp),£-\X))I{X£B n ) 



{^Y\^\h d n +2 
i 



- \\ P -£ P (v) f /2Hl {f , {p) v) ^ det{gt]){v)dv 



(47T) rf |/i| J Rd 



-M72/f 



{f(p),v) 2 dv 



+ 



l - 2 (o(^+ 4 ) + 0(^+^4 ( i 0g/l -i)(<m)/ 2 



/4 + 



we have that, taking L 2 /4 > 2 + d, 



(4.11) 



lim £Z 2 ! = s 2 



Therefore, by Lyapunov's theorem (e.g., Q, p. 44), in order to show that 
(4.12) C(Z nA )^N(0,s 2 ), 
it suffices to prove that 



(4.13) 



rE 



K 



p-X 



(f'( P ),£ p 1 (X))I(X eB n ) 



-EK 



p-X 



(f( P ),£~ 1 (X))I(X eB n ) 
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By the hypothesis on h n and (3.11) we can ignore the expected value within the 
square bracket, and for the rest, proceeding as usual, we have 



1 



nhT +2) 



E 



K 



P-X 



{^y\n\nhT +2) 



< 



2ht+ 4 



(47r) 2 |/i|n/i„ 



2(d+2) 



(f'(p),S-\X))I(XeB n ) 

e -(M 2 +o(M 4 )/^ (f(p) 7 v)\l + 0{\v\ 2 ))dv 

e-W 2 / 2 \f'(p)\ 4 dv = O(l/(nh d n ))^0, 

R d 



proving (4.13), and therefore, the limit (4.12). Now the theorem follows from Propo- 
sition 2.1, (4.5), (4.8), (4.9) and (4.12). □ 

This result extends without effort to the CLT uniform in /, which is the main 
result in this section. 

Theorem 4.2. Let U be a normal neighborhood ofp and let T be a class of functions 
uniformly bounded up to the third order in U. Assume nh n +A — ► 0, and nh^ — > oo. 
Then, as n — > oo, the processes 



(4.14) 



iht+ 2 



A/. n ,„/(p) ~ ^-A M f(p)] : / e T 



converge in law in £ 00 (^ r ) to the Gaussian process 



(4.15) 



G(f) ~ 



where Z = (Z\, . . . , Zj) is the standard normal vector in T p (M) (= R d ). 

Proof. The proof of Proposition 4.1 applied to / = J2j=i a jfj> with fj G shows 
that the finite dimensional distributions of the processes (4.14) converge to those 
of the process (4.15) (by the definition (4.3) of s = s(f)). Also, by Theorem 3.1, 
we can center the processes (4.14). Hence, the Theorem will follow if we show that 
the processes Z n = Z n (f) in (4.4) are asymptotically equicontinuous with respect 
to a totally bounded pseudometric on T (e.g., [a]). 

First, by the computation (3.8) we can restrict the range of Xj to Xi £ B n by 
taking i 2 /4 > d + 3, because 



V nh 



d+2 



E sup 

ft? 



EK 



< 



4cnhn ^ 4 



d+2 



p-Xj 

h n 

p-x 

h n 

+ 0, 



(/(*i)-/(p))/(-XieM\B B 
(f(Xi)-f(p))i(XeM\% 



since nh c l +A — > 0. 
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Now that we can restrict to Xi G B n , we only need to consider Z n ^(f), i = 1,2, 3, 
as denned by equations (4.10), (4.6) and (4.7) respectively. Asymptotic equiconti- 
nuity of Z n ^(f) follows because 



E sup \Z nA (f)\ 2 <S 2 E 

ll/'(p)ll<« 



vnhjF^v v fen y 



<5 2 



A' 



< 



(4tt)<V|/4 +2 7g„ 
<5 2 



(4tt)<V| Jn" 



i e -\\v-£p(v 
e-WM 2 dv 



£-\X)I{X&B n ) 



\v\ 2 ^1 det(gij)(v)dv 



which tends to zero when we take sup over n and then limit as S — > 0. 
Next, by the computation in (4.9), 



Esup\Z nt3 (f)\ =o(^K) -0. 



Finally we consider Z n ^{f)- Let Si be i.i.d. Rademacher variables independent 
of {Ai}. Then, by symmctrization, 



£supZ 2 !2 (/) 



< 



P-Xj 



/"( P )(£:- 1 (A 4 ),f- 1 (A 4 ))/(A 4 eB n ) 



Next, we recall that, for an operator A in R d , (or in T P (M)), we have the following 
identity for its quadratic form 

A(u, v) := (Au, v) = (A,u® v)hs, 

where in orthonormal coordinates, u(£> v is the c? x e?-matrix with entries UiVj, and 
the Hilbcrt-Schmidt inner product of two matrices is just the inner product in R 2d . 
Also note in particular that ||u<g> v\\hs — \ u \\ v \- Therefore, 



E e sup 



^ £t A('^ 7 -^Yr(p)(£- 1 (A l ),£- 1 (A l ))/(A, G B n ) 

i—l ^ n / 



£ £ su P (/»,£ 



e,K 



=i 



P -^\ {£;\Xi) ® £- 1 (A,))/(A, G B„) 



<£; e sup|/"(p)| 2 ffS i? £ 



£ £i tf(L_^ (£-!(*.) ® £-i(^))/(A, G B„) 



< 



P_A« 

/in 



IKf" 1 ^) ® f" 1 ^))!! G B n ) =: c 2 A 2 . 
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Now, by (4.8), 

EAl=nEK^'^j\\£-\X)fl(XeB n )<^^ 1 I e^^dv, 



which gives 



EsupZi 2 (f)=O(hi)^0. n 

A simpler proof along similar lines gives the following law of large numbers: 

Theorem 4.3. Let U be a normal neighborhood ofp and let T be uniformly bounded 
and equicontinuous up to the second order in U. Assume h n — > and nh^ +2 — > oo. 
Then 



(4.16) sup A hn , n f(p) - iA M /(p) 



in pr. 



A Law of the Iterated Logarithm is also possible, but we refrain from presenting 
one since in the next section we will give a law of the logarithm for the sup over 
/ e T and p e M, and the same methods, with a simpler proof, give the LIL at a 
single point. 



5. Uniform approximation of the Laplacian by graph Laplacians 

This section is devoted to results about approximation of the Laplacian by graph 
Laplacians not only uniformly on the functions /, but also on the points p G M, 
M a compact submanifold or M as in Remark 3.2. The distributional convergence 
requires extra work (recall the Bickcl-Rosenblatt theorem on the asymptotic distri- 
bution of the sup of the difference between a density and its kernel estimator) and 
will not be considered here. 

Although the results in this section are also valid in the situation of Remark 3.2, 
we will only state them for M a compact submanifold (without boundary). Also, 
we will identify M with </>(M), that is, the imbedding <j> will not be displayed. 

Theorem 5.1. Let M be a compact Riemannian submanifold of dimenison d < m 
o/R m , let X,Xi be i.i.d. with law [i/\fi\ and let K be as defined in (3.1). Let T be 
a class of functions uniformly bounded and equicontinuous up to the second order 
in M. If h n — > and nhf^ 2 / \ogh~ 1 — > oo, then 



(5.1) 



sup sup 



A hn , n f(q) - |-r A M /(g) 
IMI 



a.s. 



as n — > oo. Moreover, if J 7 is a class of functions uniformly bounded up to the third 
order in M, and, in addition to the previous conditions on h n , nhf^ 4 / log h~ x — > 0, 
then 



(5.1') sup sup A hnjn f(q) - —A M f(q) 
feFqeM |MI 




log(l/ft„ 



7 d+2 

nn n 



as n 



oo a.s. 



Proof. By Remark 3.1 on Theorem 3.1 (more precisely, by its uniform version), in 
order to prove (5.1) it suffices to show that 



(5.2) 



sup sup \A hn!n f(q) - A hn f(q)\ 

fGFq€M 



a.s. 
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Let 0„ >g = {x <E M : ||a; — g|| < L/i n (log/i„ 1 ) 1 / 2 }, where, we recall, || • || is the norm 
in R m .' Then, as in (3.8) and (4.5), if L 2 /4 >d + 2, 



sup sup 

/ 9 



£ U((q - XMhJIiXi G B^ifiXi) - /(g)) 
z ™ i=i 

(5-3) - - X)/K)I{X G B^ q )(f(X) - /(<?)))) 

< 4||/||ooftn 2/4 n 

- /4 +2 

To establish (5.1), we show that 
1 n 

£ « := ^+2 Ssu p su p E ( K (b - x i )/h n )i(x l e B n , q )(f(Xi) - /(g)) 

(5.4) 

- - X)/h n )I(X G B n , q )(J(X) - /(<?))) - 0, 

and use Talagrand's [14| concentration inequality to transform this into a statement 
on a.s. convergence. 

Each function / G T can be extended to a twice continuously diffcrcntiable 
function / on a compact domain N of R m with M in its interior such that the 
classes {/ : / G J 7 } {/ : / e JF} {/ : / G -T 7 } arc uniformly bounded and 
{/ : / G J* 7 } is equicontinuous on ./V (use a finite partition of unity to patch 
together convenient extensions of / in each of the sets in a finite cover of M by e.g., 
geodesic balls: see e.g. Lee [9], pp. 15-16). Then, 

f(X t ) - f(q) = J\q + - q))(X t - q) 

for some point < 9 = q> Xi < 1- 

Note that, M being compact, B niq is contained in one of a finite number of 
uniformly normal neighborhoods for all n > No, with No < oo independent of q, 
so, we can use q-normal coordinates and notice that for these coordinates, on B n , q , 
we have the inequalities (3.10) holding uniformly in q (by Proposition 2.2). Since 
the derivative / is uniformly bounded, for n > N\ (independent of q), we have 

EK 2 {(q - Xi)/h n )I(Xi G B n , q )(f(Xi) - /(g)) 2 

< CEK 2 {(q - Xi)/h n )I(Xi G B nA )\\Xi - q\\\ 

which, in view of (3.10), can be further bounded by 

2CEK 2 ({q - XiyhnWXi G 6„, g )|f- 1 (X 4 )| 2 

<2C [ e -W 2 - c ^ 2h «\v\ 2 (l + CM 2 )dv 

< 2Ch d n + 2 f e-l l a / 3 |i;| 2 (l + d\v\ 2 )dv < C 2 h d + 2 , 

so we have 

(5.5) EK 2 ((q - Xd/hJUXt G B ntq ){f(X t ) - f(q)) 2 < C 2 h d + 2 



with a constant C 2 that does not depend on q. 
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To prove (5.4), we replace f(Xi) — /(g) by its Taylor expansion of the second 
order: 

fiX,) - /(g) = f{q){Xi - g) + -/' ' (q){X t - q ,X i -q) + r n (f; g; X,), 

where 

sup supr n (f;q;X) < S n \\X - q\\ 2 

with 5 n — » as n — ► oo (because of equicontinuity of {/ : / € JF} and the fact 
that || JC - q\\ < Lhnilogh- 1 ) 1 ' 2 -> 0). 

The first order term leads to bounding the expectation 

sup sup 



[f'(q), E - Xi)/hn)I(Xi G B»,«)(*< - 3) 

i=l 

- £X((g - X)/h n )I(X G B„, g )(X - g) 

which is smaller than 

sup sup II V (V((g - Xil/hJIiXi G S», g )(JQ - g) 
- - X)/h n )I(X G B„ >8 )(X - g) 



6 



where 6 is a uniform upper bound on / . Denote the coordinates of x G R m in 
the canonical basis of R m by x a , a = 1, . . . , m and consider the class of functions 
M h-> R, 

^ = {f q , h ,x(x) ■= e-^ 2 / ih2 I(\\x - g|| < A)(z Q - g„) : q G M, /i > 0, A > 0}. 

By arguments of Nolan and Pollard (1987), the class of functions of a;, {e _ ll 9_:E ll / ih : 
q G M, h > 0} is VC subgraph; and it is well known that the open balls in R m are 
VC and that the class of functions {x a — q a : q G M} is also VC subgraph (see, 
e -g-; 01)- The three classes are bounded (resp. by 1, 1 and 2sup{||x| : x G M}) and 
therefore, by simple bounds on covering numbers, the product of the three classes 
is VC-type with respect to the constant envelope C = 2 + 2sup{||ir| : x G M}. 
In particular, if N(Q,e) are the covering numbers for Q in L2 of any probability 
measure, then 

N(G,e)<(jy 

for some A, v < 00 and all s less than or equal to the diameter of Q . Hence, by 
inequality (2.2) in Q, there exists a constant R such that 

n 

Esupl V (Kfa-XiyhJUWXi -q\\ < LhnQogh-^XXi.a-qc) 
1 i=i 

(5.6) - E(K{{q- X)/h n )I{\\X - q\\ < Lh n (logh^ 2 )(X a - q a ) 



— A A 
< R( \/na\jlog — V log — 
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where a 2 > supj g g Ef 2 (X). Now, to compute a we use again our observations 
before the proof of (5.5). For n > N\ (independent of q), we have 

S upEf(X)< [ e -(l"l a - c H 4 )/ 2fc »| v | 2 (l + Ci|i;| 2 )d« 

< K+ 2 ( e^ 2 / 3 \v\ 2 dv < C 2 hf+ 2 , 

JTL d 

for some C 2 < oo independent of q. So, we can take a 2 — C2h d l +2 . Hence, by the 
hypothesis on h n , the right hand side of (5.6) is bounded by 

for some R' < oo. 

To handle the second order term, note that 

1 ™ 

— ^Ssupsup V (K((q - Xd/hJIiXi € B n , q )f(q){Xi -q,X t - q) 
nn n f q i , =1 v 

- EK((q - X)/h n )I(X e B n , q )j"(q)(X -q,X- g)) 
E sup sup I (j"(q), 



nh d n + 2 f g 

n 

(K((q - X t )/h n )I{X t E B n , q )(Xi - q) ® (X, - g) 

- BA-((g - X)/h n )I(X e B n ,,)(X - g) ® (X - g)) \ 

which is dominated by 
b n 

— ^Ssupsup II V (^((g - Xi)//i„)7(Xi e B n , q )(Xi - g) ® (X - g) 
- £tf((g - X)/h n )I(X e B n , g )(X - q) ® (X - g)) 

(with 6 being a uniform upper bound on /"). Here <g> denotes the tensor product of 
vectors of R m and || • \\hs is the Hilbcrt-Schmidt norm for linear transformations 
of R m . This leads to bounding the expectation 

n 

Esup | ( K ((l - X^/h n )I(\\X t - q\\ < Lhnilogh- 1 ) 1 / 2 )^ - q a )(X hP - q fj ) 

(5.6'] 1=1 , 

- E[K{{q - X)/h n )I(\\X - q\\ < Lh n (log h' 1 ) 1 ' 2 )^ - q a ){X - q ) 

for all 1 < a, (3 < m, which is done using the inequality for empirical processes 
on VC-subgraph classes exactly the same way as in the case (5.6). This time the 
bound becomes 

A \i/2 



i?'(<+ 4 logi_) ] 
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For the remainder, we have the bound 

1 I " 

— m Esupsvip\Y / K((q- X^/h^IiX, G B n , q )r n (f,q,Xi) 
f q 1 i=1 

- EK((q - X)/h n )I(X G B n , q )r n {f, q, X)) 



< — ^Esup V - Xi)/h n )I(Xi G B^jJHXi - g|| 2 



n/4+ 2 



sup*; (#((<? - xyhjiix g b«,,)||a: - ?|| : 



< — ^2^su P J] - Xi)/h n )I(Xi e B thq )\\X t - 



2S n 

,d+2 



- EK((q - X)/h n )I(X G B n , q )\\X - q\\ 
supEK((q - X)/h n )I{X G B n , g )||X - g|| 2 . 



The first expectation is bounded again by using the inequality for VC-subgraph 
classes and the bound in this case is 



(n^+ 4 log 



ihi+ 



h„ ) V nhi 



The second expectation is bounded by replacing \X — g|| 2 by \£ ~ 1 (X)| 2 and chang- 
ing variables in the integral (as it has been done before several times) . This yields 
a bound of the order C5 n , which also tends to 0. 

Combining the above bounds establishes (5.4). One of the versions of Talagrand's 
inequality (e.g., [l(|) together with (5.5) gives with some constant K > and with 
probability at least 1 — e~ l 

1 n 

- Sup sup ]T (K((q - Xi)/hn)I{Xi G B n , q ){f{Xi) - f(q)) 

- EK((q - X)/h n )I(X G B n , q )(f(X) - /(g))) 

<K(h d + 2 E n + Jh?+ 2 - + -). 
\ V n n ) 

Taking t := t n := Alogn with large enough A, so that J2 n e ~*" < 00 an< ^ usm § 
Borel-Cantelli Lemma shows that a.s. for large enough n 



^ sup sup I £ (Kdq-XiyhnWXi G B n , q )(f(Xi) - /(«)) 

; " / ^ i=l 

- ^((g - X)/h n )I(X G B n , q )(f(X) - /(g))) 



< if £7, 



' ^4 log n A log n 



d+2 



d+2 



and since, in view of (5.4) and under the condition nhf^ 2 / log /i^ 1 — > oo, the right 
hand side tends to 0. This and (5.3) yield (5.1). (Note that nh d+2 / log /i^ 1 — > oo 
implies nh^ +2 / log n — > oo.) 
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The proof of (5.1') requires the following version of Taylor's expansion of / : 

f(Xi) - f{q)=l'(q)(X l -q) + \f (q){X % - q, X t - q) 

+ ~7"'(« + Oi(Xi - q))(Xi -q,Xi- q,X, - q). 
6 

The first two terms have been handled before, and the expectations of the sup- 

/, i 

norms of the corresponding empirical processes were shown to be 0(w ° g h d+2 )■ The 
third term leads to bounding 



1 71 
— S sup sup I ^ (K((q-Xi)/h n )I(Xi G B n<q ) 

nh n f q I i=1 V 

- EK({q - X)/h n )I(X G B n<q ) 
x \T\q + Oi(Xi - q))(Xi -q,X t - q, X x - q) 

which, for /"' uniformly bounded by b, is smaller than 
b 



^£su P ]T (K((q - Xi)/h n )I(Xi G B„, q )\\Xi - q 
bn 



Gnh' n q — 1 



— m sup EK((q - X)/h n )I(X G B n>q )\\X - q 
bnti n q 



b n 

^ 7TTm Esn V I E ~ Xi)/h n )I{Xi G B„, g )||Xi - 9 || 

nnh„ q I — \ 



- EK((q - X)/h n )I(X G S n , g )||X - 9 || 
+ — ^ supEK((q - X)/h n )I{X G B„, g )||X - 9 j| 3 , 

which can be handled exactly as before and shown to be of the order 



/log hn 1 ( /log /in 1 



by the conditions on /i„. Using Talagrand's inequality the same way as before, 
completes the proof of (5.1'). □ 

We conclude with the following theorem, whose proof is a little longer and more 
involved, but it is based on a methodology that is well known and well described in 
the literature (see i and 0). Its extension to the case of manifolds requires some 
work, but is rather straightforward. 

Theorem 5.2. Let M be a compact Riemannian submanifold of dimenison d < m 
o/R m , let X,Xi be i.i.d. with law and let K be as defined in (3.1). Assume 

that h n — > 0, nh n +2 / log h~ x — > oo, and nhf^ 4 / log h^ 1 — > 0. Let J 7 be a class of 



Laplace- Beltrami operators 
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functions uniformly bounded up to the third order in M. Then, 




s vPfer, q eM (EJ=i (!£(?)) ) 



where x 
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